1. Field of the Invention
This invention relates to attitude estimation and motion compensation for real-time image processing on a moving platform, possibly in combination with real-time active stabilization of the image sensor.
2. Description of the Related Art
Image sensors are mounted on moving platforms such as missiles, aircraft, terrestrial vehicles, ships, spacecraft or hand-held devices such as video cameras, binoculars etc. The image sensor may be either fixed to the moving platform or mounted on a single or multi-axes gimbal. In many applications, the sensor line-of-sight (LOS) must be both measurable and controllable in real-time. The moving platform, gimbal or sensor optics may be actively stabilized in real-time to, for example, maintain the sensor LOS on a target.
An Inertial Measurement Unit (IMU) that is rigidly mounted to the moving platform provides discrete measurements of angular rate of motion of the platform. These measurements are integrated into raw attitude estimates (e.g., yaw-pitch-roll Euler angles, direction cosine matrix, quaternions, or some equivalent). The IMU, hence the raw attitude estimates S(hT) where h is an integer clock index and T is the sampling period, will exhibit a latency L of a few to several clock samples, and not necessarily an integer multiple. The raw attitude estimate for the current sample has an error associated with this latency, the greater the latency the greater the error on average. Because the moving platform, gimbal and/or optics are actively stabilized in real-time at the sampling rate of the IMU, the active stabilization system cannot wait for the latent raw attitude estimate.
To reduce the error (on-average) of the attitude estimate for the current time sample, systems implement a prediction filter that extrapolates forward in time to compensate for the latency of the IMU measurements. The prediction filter processes a trailing window of the raw attitude estimates up to the current sample and extrapolates forward in time to generate a causal attitude estimate A(hT|hT). The causal attitude estimate A(hT|hT) reads as the estimate of attitude A at discrete time hT given all information available up to time hT. The prediction filter is and must be “causal” in the sense that only raw attitude estimates S(hT), S((h−1)T), S((h−2)T) . . . available at the current time sample (hT) can be processed to generate the attitude estimate A(hT|hT) for the current time sample hT. Again the active stabilization system cannot wait to accommodate the latency of the IMU. U.S. Pat. No. 7,729,816 describes an example of a causal prediction filter, referred to therein as a “lead filter” for active stabilization of a sensor LOS on a satellite.
In many of these same applications, the images captured by the image sensor are motion compensated in real-time to account for changes in sensor LOS. Motion compensation consists of using attitude estimates to compensate (e.g. shift) the images to compensate for relative motion of the sensor LOS from image-to-image so that the sensor can properly reconstruct the true inertial LOS to a distant target. For example, a stationary target in a scene may appear to move due to changes in sensor LOS. Motion compensation removes this non-scene motion. Real-time motion compensation may be performed to support closed-loop applications such as target acquisition, identification or tracking, to accommodate limited memory or processor resources or to stream the video signal to a memory device. The active stabilization system provides the causal attitude estimate A(hT|hT) to the image processing system, which is down sampled to the imaging rate (A(kMT|kMT) where M is the down sampling factor, to perform the motion compensation. On average, the error of the predicted attitude estimate A(hT|hT) will be less than the error of the raw attitude estimate S(hT) at current time sample hT.